E cient Estimation of Mutual Information for Strongly Dependent Variables
نویسندگان
چکیده
We demonstrate that a popular class of nonparametric mutual information (MI) estimators based on k-nearest-neighbor graphs requires number of samples that scales exponentially with the true MI. Consequently, accurate estimation of MI between two strongly dependent variables is possible only for prohibitively large sample size. This important yet overlooked shortcoming of the existing estimators is due to their implicit reliance on local uniformity of the underlying joint distribution. We introduce a new estimator that is robust to local non-uniformity, works well with limited data, and is able to capture relationship strengths over many orders of magnitude. We demonstrate the superior performance of the proposed estimator on both synthetic and real-world data.
منابع مشابه
Efficient Estimation of Mutual Information for Strongly Dependent Variables
We demonstrate that a popular class of nonparametric mutual information (MI) estimators based on k-nearest-neighbor graphs requires number of samples that scales exponentially with the true MI. Consequently, accurate estimation of MI between two strongly dependent variables is possible only for prohibitively large sample size. This important yet overlooked shortcoming of the existing estimators...
متن کاملWavelet Based Estimation of the Derivatives of a Density for m-Dependent Random Variables
Here, we propose a method of estimation of the derivatives of probability density based wavelets methods for a sequence of m−dependent random variables with a common one-dimensional probability density function and obtain an upper bound on Lp-losses for the such estimators.
متن کاملMine: Mutual Information Neural Estimation
We argue that the estimation of the mutual information between high dimensional continuous random variables is achievable by gradient descent over neural networks. This paper presents a Mutual Information Neural Estimator (MINE) that is linearly scalable in dimensionality as well as in sample size. MINE is backpropable and we prove that it is strongly consistent. We illustrate a handful of appl...
متن کاملHydrograph Estimation based on Various Components of Rainfall Using Adaptive Neuro-Fuzzy Inference System in Kasilian Watershed
Flood hydrograph preparation and estimation are considered a comprehensive information for soil and water managers and planners. While it is not simply possible preparing it for all watersheds. Therfore suitable flood hydrograph estimation and modeling seems to be necessary using available rainfall data. The study area is located in Kasilian representative watershed in Mazandaran province compr...
متن کاملEstimation of the Survival Function for Negatively Dependent Random Variables
Let be a stationary sequence of pair wise negative quadrant dependent random variables with survival function {,1}nXn?F(x)=P[X>x]. The empirical survival function ()nFx based on 12,,...,nXXX is proposed as an estimator for ()nFx. Strong consistency and point wise as well as uniform of ()nFx are discussed
متن کامل